feat: attic cache support - #671
Conversation
|
I'm not great with haskell btw, so I would greatly appreciate the feedback |
|
attic-client needs to be added to hercules-ci-agent PATH: services.hercules-ci-agent = {
enable = true;
package = inputs'.hercules-ci-agent.packages.hercules-ci-agent;
};
systemd.services.hercules-ci-agent = {
path = [ pkgs.attic-client ];
}; |
Looks great to me, but depending on your preference I could either
Whichever you prefer :) |
|
Encountered one problem: after removing cachix from the system and performing gc, hercules thinks that the package is built and machine can fetch it from the cache, but it isn't, and it doesn't work properly, I wish there was an ability to make hercules-ci forget about that/hercules-ci was first re-checking presence of the paths in the cache This also makes testing complicated |
1790727 to
3b01d6f
Compare
cdda5b0 to
0abaeed
Compare
| <> (atticCaches & toList <&> AtticCache.publicKeys & join) | ||
| netrcFile <- Netrc.getNetrcFile | ||
| Netrc.appendLines netrcLns | ||
| Netrc.appendLines (netrcLns <> Attic.toNetrcLines atticCaches) |
There was a problem hiding this comment.
Untested for now, I'm also not sure why cachix handles it in a different way
b05f655 to
8f91da7
Compare
| [("XDG_CONFIG_HOME", cfgDir), ("XDG_CACHE_HOME", cfgDir)] | ||
| ++ filter (\(k, _) -> not (isXdg k)) oldEnv | ||
| let args = | ||
| ["push", "--no-closure", toS (AtticCache.cacheName cache)] |
There was a problem hiding this comment.
Also given the described caching problem it seems I should not rely on --no-closure here.
I was assuming hercules will build paths one-by-one, and I do not need to calculate closure in this case, but given its behavior in the case of gc - I should not rely on this assumption (it tries to rebuild dependencies as a part of derivation; I think something like build-one-derivation-but-only-substitute-its-inputs should be added to nix for this case)
8f91da7 to
ba8d6b6
Compare
|
That's funny, with default configuration attic has managed to DoS itself... https://hercules-ci.com/github/deltarocks/jrsonnet/jobs/2 I would need to reconfigure attic instance used for hercules. That is after I have switched it to push closures instead of the specific derivations. To see the pushing progress, maybe attic stdout should somehow be piped to hercules logs... |
2 participants
Pulls through the standard nix binary cache interface, pushes through attic own CLI - given that it uses its own protocol for that, I don't think there is an easier solution.
Only works for public caches for now, because attic uses path-based routing, and the existing implementation uses netrc which only allows per-host secrets, I'm not sure what would the best way to add it as-is, I have troubles understanding some of the logic here.
Fixes: #654
Tested on https://github.com/CertainLach/jrsonnet CI